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TITLE  CONT.:  Gaussian  Channel 


Abstract 


Information  capacity  is  determined  for  the  additive  Gaussian  channel 
when  the  constraint  is  given  in  terms  of  a  covariance  different  from  that 
of  the  channel  noise.  These  results,  combined  with  previous  results  on 
capacity  when  the  constraint  covariance  is  the  same  as  the  noise  covariance, 
provide  a  complete  and  general  solution  for  the  information  capacity  of 
the  Gaussian  channel  without  feedback.  They  are  valid  for  both 
continuous-time  and  discrete-time  channels,  and  require  only  two  assumptions 
the  noise  energy  over  the  observation  period  is  finite  (w.p.  1),  and  the 
constraint  is  given  in  terms  of  a  reproducing  kernel  Hilbert  space  norm. 
Applications  include  channels  with  ambient  noise  having  unknown  covariance, 
and  channels  subject  to  jamming.  The  results  for  the  mismatched  channel 
differ  markedly  from  those  for  the  matched  channel. 


Introduction 


The  information  capacity  of  the  Gaussian  channel  without  feedback,  subject 

to  a  generalized  energy  constraint,  is  determined  in  [1],  In  that  work,  the 
constraint  is  given  in  terms  of  the  covariance  of  the  channel  noise  process. 

However,  there  are  many  situations  where  one  may  wish  to  determine  capacity 
subject  to  a  constraint  determined  by  a  covariance  that  is  different  from  that 
of  the  channel  noise.  Examples  are  jamming  or  countermeasures  situations,  or 
when  there  is  insufficient  knowledge  of  the  natural  environment. 

Qiannels  where  the  covariance  of  the  noise  is  the  same  as  that  of  the 
constraint  will  be  called  matched  channels;  otherwise,  the  channel  is  said  to 
lie  mismatched  (to  the  constraint).  In  this  paper,  the  capacity  of  the  mis¬ 
matched  Gaussian  channel  is  determined.  Results  for  a  restricted  class  of 
mismatched  channels  are  given  elsewhere  [2].  Various  special  cases  of  the 
mismatched  channels  have  been  treated  previously  [3]  -  [5], 

The  results  for  the  mismatched  channel  differ  significantly  from  those 
for  the  matched  channel.  A  discussion  of  these  differences  follows  the  proof 
of  the  main  result. 

.•\n  example  of  the  type  of  problem  to  which  the  results  given  here  apply 
is  the  following.  Suppose  that  one  wishes  to  obtain  the  information  capacity 
of  the  additive  Gaussian  channel  with  output 

Yft)  =  Jq  (B[X])(s)ds  +  N(t),  t  in  [0,T] 

where  (X(t))  is  the  message,  B  is  a  coding  function,  {N(t))  is  zero-mean 
Gaussian  noise  independent  of  the  message  (Xft)),  and  the  constraint  is 

}■.  [B[Xl(t)]^dt  <  P.  The  solution  to  this  problem  is  given  in 
Proposition  1  and  Theorem  2  (if  the  process  (B[X](t))  is  restricted 
to  lie  in  a  finite-dimensional  subspacc)  and  in  Theorem  3  (if  there 


is  no  restriction  on  the  dimensionality  of  the  process  (B[X](t)). 

If,  for  example,  the  signal  detection  problm  of  N  vs.  W  is  non-singular 
(where  (W(t))  is  the  Wiener  process),  and  if  rj^(t,s)  -  min(t,s)  is  a 
covariance  function,  where  r^^  is  the  covariance  of  (N(t)),  then  the  capacity 
for  the  unrestricted  dimensionality  signal  will  be  P/2,  the  same  as  if 
(N(t))  were  the  Wiener  process. 

The  relationship  with  the  Wiener  process  arises  because  the  above  con¬ 
straint  is  given  in  terms  of  the  norm  of  the  RKHS  for  the  covariance  function 

min(t,s):  /q  y^(t)dt  =  ||y(|^  when  y  is  in  12(0, T]  and  ||•||^^  is  the  norm 

of  the  reproducing  kernel  Hilbert  space  for  min(t,s).  When  the  detection 
problem  N  vs.  W  is  singular,  then  the  capacity  can  be  smaller  than,  equal  to, 
or  larger  than  P/2.  The  expression  for  the  capacity  will  depend  on  the 
covariance  of  CN(t})  and  the  value  of  P.  This  dependence  of  the  expression 
for  the  capacity  on  the  value  of  P  does  not  arise  when  the  channel  is  matched; 
that  is,  when  the  constraint  is  given  in  terms  of  the  norm  of  the  RKHS  of 
the  channel  noise  (N(t)).  Another  major  difference  arises  in  this  problem 
when  the  signal  process  is  not  constrained  to  lie  in  a  finite-dimensional 
subspace.  For  the  matched  channel,  the  capacity  then  cannot  be  attained; 
for  the  mismatched  channel,  it  can  be  attained  in  some  situations  and  not 
attained  in  others,  depending  again  on  the  covariance  of  (N(t))  and  the 
value  of  P.  In  this  example,  it  can  sometimes  be  attained  if  rjj^(t,s)  -  min(t,s) 
is  not  a  covariance  function;  otherwise,  it  can  never  be  attained. 


Definitions  and  Structure 


The  channel  to  be  considered  is  the  independent  additive  Gaussian 
channel  without  feedback.  The  channel  output  is  Y  =  A(X)  +  N,  where  N  is 
the  Gaussian  noise,  X  is  the  message  process  (independent  of  N),  and  A(X) 
is  the  transmitted  signal.  Hie  mathematical  structure  is  defined  below, 


as  in  [1] . 

The  message  X  is  represented  by  a  probability  (measure)  on  a  measurable 
space  (H^,  E  [Hj^]),  where  E  [Hj^]  is  a  a-field  of  subsets  of  Hie  noise 
\  is  represented  by  a  probability  Uj,  on  a  measurable  space  (H2,  B[H2]).  Hie 
transmitted  signal  A(X)  is  defined  by  a  E[Hj^]/E[H2]  measurable 
coding  function  A  from  into  H2.  The  received  signal  (channel  output)  Y 
is  represented  by  the  probability  py  (^2’  since  Y  =  A(X)  +N^ 

Py((;)  =  p^®u^{(x,n):  A(x)  +  n  e  Cl  for  C  in  E[H^] ,  being  the  product 


probability. 


The  channel  probability  on  the  product  measurable  space 


(ll^xii2,  E[llj^xil2l)  is  defined  by  “  yj^«yj^,{  (x,n) :  (x,A(x)+n)  e  C)  for 

C  in  E[Hj^xH2]  .  The  average  mutual  information  is  then  Ify^^-l,  where  UMyyl  - 


if  it  is  false  that  y^  is  absolutely  continuous  with  respect  to 


.j^SUy  (py^,  «  y^Spy) ,  and  otherwise 


■  ^X\'^ 


H^xH2 


(x,y)duy^.  (x,y). 


'fhe  information  capacity  is  then  sup  I  [Pwl  >  where  Q  is  a  set  of 

Q 

admissible  pairs  (y^,A). 

and  H2  will  be  taken  as  real  separable  Hilbert  spaces  with  E[H|]  the 
Borel  a-field  of  H^.  <•  •>j  will  be  the  inner  product  for  Hj^,  <*,•>  the  inner 
product  for  H2,  ll*|||  and  ||  •  ||  the  corresponding  norms.  Pj^  will  be  assumed  to  have 
zero  mean  and  finite  second  moment:  |!x||  dyj^(x)  <  0°.  For  H2  =  L2(0,T)  or 


this  corresponds  to  an  assumption  o£  finite  energy.  In  these  cases,  pjj  is 
induced  by  a  path  map  from  an  underlying  probability  space 
iij^rCC)  =  P{w:  N(u))  e  Cl  for  C  in  where  (N^)  is  a  measurable  stochastic 

process  with  almost  all  paths  in  11,.  If  H,  =  L2[0,T],  and  (N^)  has  zero-mean 

z  r 

and  covariance  function  rj^,  then  HxH  dpj^(x)  =  rj^(t,t)dt. 

A  covariance  operator  in  il,  is  (here)  any  bounded  linear  operator  on 
il-,  which  is  also  symmetric,  non-negative,  and  trace-class.  A  probability 
..  on  (11-,,  1B[I12])  has  such  a  covariance  operator  if  and  only  if  p  has  finite 
sccuiul  moment;  then,  /jj  j!.x  i|“dii  (x)  ^  I'race  R  ii  R,  wheie  R  is  the 

z 

covariance  operator  of  defined  by  (assuming  now  that  y  has  zero  mean) 

Ru,v  =  x.u  ■  x,v  dnCx). 

':he  covariance  op.  r  itor  of  y^.  will  be  denoted  by  Rj^.  One  can  assume 
Wh(X;  that  rangeCl^  =  H2  [1],  so  that  is  strictly  positive  and  exists. 
For  11,  =  L,[0,T1,  R^.  can  be  represented  by  an  integral  operator  with  kenicl 
i'nnc  1 1  on  r.^ . 

A  measure  l  m  (H,,  is  Gaussian  if  for  every'  v  in  H,,  the  maj) 

x  -^<x,v'’  defines  a  Gaussian  distribution  on  R  ;  i.e.,  F(a)  =  y{x:  <x,v'-  '0 

defines  a  Gaussian  distribution.  It  is  knoivn  that  there  is  a  1:1  relationship 

between  covariance  operators  in  H2  and  zero-mean  Gaussian  measures  on  H2. 

1 

Let  R  be  a  strictly  positive  covariance  operator  in  H^,  wdth  the 

1 

positive  square  root  of  R;  range(R'^)  is  a  separable  Hilbert  space  under  the 

_  1  _  1 

inner  product  ■u,v>,.  =  -R  '^u,  R  ^v>. 

K 

1-,[0,T]  is  the  space  of  all  Lebesque-square-integrable  real -valued  func¬ 
tions  on  [0,T];  1^2  [0,1]  consists  of  the  crjuivalence  classes  formed  from 
elements  of  I-^[0,T]. 


Constraints 


Suppose  that  is  a  strictly-positive  covariance  operator  in  ll^.  If 

1 

11,  is  infinite  dimensional,  then  range(R^)  is  a  proper  subset  of  H2  and  is 
a  separable  Hilbert  space  under  the  inner  product 
<u,v>„  =  I  <u.b„><v,b„>/0.„  = 

uhere  (b^^)  are  c.o.n.  (complete  orthonormal)  eigenvectors  of  and  are 
corresponding  eigenvalues. 

If  H,  =  L2[0,T],  then  has  a  representation  as  an  integral  operator 

with  kernel  r,,,.  r,,.  can  be  defined  as  a  measurable  covariance  function  on 

'A  'A 

[0,T]x[0,T]  and  then  defines  a  RKHS 

of  functions  on  [0,T],  for  which  r^^,  is  the  reproducing  kernel,  with  inner 

product  (u,v)j^..  Let  [u]  denote  the  equivalence  class  in  L^[0,T]  defined 

1 

by  the  function  u  in  L2lO»T]*  'Hien  [u]  is  in  range(R^)  if  and  only  if  [u] 

is  generated  by  an  element  u  in  ftoreover,  (u,v)^^  =  <[u],[v]>^.  Thus, 

in  ail  that  follows,  one  can  consider  L2[0,T]  as  a  concrete  exairple  of 

1 

II,,  identify  with  range (R^^),  and  consider  H’ilw  inner 

product  and  corresponding  norm  for 

The  Wiener  process  is  frequently  used  to  model  a  white  noise  channel  by 
formally  considering  the  "integrated"  channel.  If  W  is  the  Iviener  process  on 
[0,T],  then  x  in  H^^.  has  RKHS  norm  |ix|i^^.  =  /q  [x(t)]^dt;  a  function  x  on  fO,T] 
belongs  to  if  and  only  if  x  is  absolutely  continuous,  vanishes  at  the  origin, 

and  has  derivative  in  ^[0,T].  In  modeling  the  "integrated"  white  noise  channel 

7 

by  the  Wiener  process,  the  constraint  Ejjx||^^.  £PT  is  then  an  average  power  con¬ 
straint  on  the  original  signal  x.  Of  course,  this  constraint  has  also  been 
used  if  (W^)  is  the  actual  (not  "integrated")  channel  noise  [6]. 

A  constraint  which  is  appropriate  in  the  case  of  the  observation  time 


pia^vpi Jim 


*  ^  y  ■  V  ■  V  • 


u  »■  V  *  V.""  V  *  V  V  L  ■  L.  M 


[0,T]  when  (IV^)  is  stationary  with  spectral  density  is 


9 


6 


where  X  is  the  -Fourier  transform  of  the  function  x. 
Kelly,  Reed,  and  Root  [7]^  for  x  in  (-«>,«>), 


-00 


x(x) !‘ 


1  2 

dA  =  lim  Y  "P 


From  a  result  ol' 


where  Xp  is  the  restriction  of  x  to  [0,T]  and  ^  is  the  RKHS  norm  of 

W  restricted  to  [0,T].  If  (X^)  is  also  stationary  with  spectral  density  fg, 
then  with  additional  assumptions  one  has 

limi  E|!)C1|2  t  =  4  r  [fsM/f„(X)ldX. 

'J'-KO  *  -00 

r.n  (general, 

liini  FllXpll^.^^  > 

An  appropriate  constraint  is  thus 

\  I|a(x)||2  <  P  (A-i) 

X 

where  R^,  is  a  covariance  operator  in  H2.  This  constraint  will  be  used  in 
this  paper;  no  other  assumptions  will  be  made. 

If  R^  is  not  strictly  positive,  then  the  constraint  A-1  can  still  be 
used  after  replacing  H2  with  range  (.R^^).  One  may  thus  suppose  WLOG  that 

H2  =  range (P^) ,  so  that  is  strictly  positive. 


where  x  >0  for  n^l  z  t  <  «>,  {u  ,  n>l}  is  a  c.o.n.  set  and  (u®v)x  =  <v,x>u. 
n  ’  n  n  n’  v  ^  > 

hhen  has  (1)  for  covariance  and  is  Gaussian  then  [1] 

I[Uxy1  =  ('^'1 

The  constraint  A-1  can  be  written  as 

l;  ||r;'"A(X)  ||-  -  Trace  i 

"ITie  supremum  of  (2)  subject  to  the  constraint  (3')  is  the  capacity 
sought  and  will  be  denoted  as  C^^.(P) ;  the  capacity  for  the  matched  channel 
fR^^.  =  Rj^)  will  be  denoted  by  Cj^(P) . 

I5  V- 

Proposition  1:  C^^(P)  is  finite  if  and  only  if  range(R^:)  rangeCRj^).  This  is 

equivalent  to  the  existence  of  a  densely-defined  self-adjoint  linear  operator 
S  in  11^,  as  follows: 

f] )  S  =  U(I  +  V)'V  -  I 

where  U  is  unitarx',  V  is  bounded  and  self-adjoint,  I  +  V  is  strictly- 
positivc,  and  R^^.  =  R^(I  +V)'^U*; 

(2)  I  +  S  is  strictly  positive  and  bounded  away  from  zero; 

(31  RjJ  =  U(I +S)'''^R^..  (A-. 

Proof:  Range (R^^rl  c  range (R^.)  if  and  only  if  there  exists  a  bounded  linear 
operator  V  in  such  that  R^^.  =  Rj^,CI  +  V)R^[8] .  This  is  equivalent  to 

^  V)'^*  for  U  unitar>'  in  H2.  The  constraint  (A-1)  is  satisfied  if 

t  v- 

and  only  if  R^  =  for  C  trace-class  with  trace  C<P.  Then,  if 

range (Rjp  c  range (R^.),  =  R^(i  +V)%*aJ(I  +  V)'^R^.  for  bounded  V  and  unitarv- 

1-  V- 

II.  The  operator  (I  +  \1  ^U*CU(I  +  V'l^  is  then  trace-class,  vcith  trace  bounded 
above  by  |1I+V||P.  Prom  [1],  C^^.(P1  <  ||I+V||P/2. 


(.ionversely,  suppose  tliat  =  K  •  «>.  Then  every  admissible 

must  satisfy  TR^  for  trace  T  <K  [1].  This  implies  A(X)  is  in 

1-  t- 

range (R^)  a.e.,  [9].  If  range (R^)  is  not  contained  in  range (Rj^),  then 

I  I  2 

there  exists  z  in  range (R^),  z  =  R^,  |lul|  =  P,  with  z  not  in  range (R^). 

Let  A  = I  and  take  Gaussian  with  covariance 
satisfies  (A-1)  but  A(X)  lies  outside  range(R^)  with  probability  one.  Thus, 
Cj^CP)  finite  implies  rangeCR^)  c  rangeCRj^). 

The  above  proof  shows  that  Cy^(P)  is  finite  if  and  only  if  there  exists 


=  zoz.  Then  (A, 


a  bounded  self-adjoint  operator  V  with  R^^=  R^.(I  +  with  U  unitary  in  H2. 

Guppose  that  R^^,  has  such  a  representation.  Since  U  is  unitary  and  R,^ 
strictly  positive,  (I  +  V)'^  must  exist.  Let  S  =  U(}  -  I,  so  that 

I  +  S  =  Ufl  +V)"V,  (I  +  Sj^  =  U(I  +  Then,  since 


=  R^(I  +  =  U(i  +  it  follows  that  Ufl  +  =  U  RjJ  =  Cl  +  S)  "R^^, 

so  that  (I +  S) ^  is  defined  on  the  dense  linear  manifold  range (R^),  and 
II  (I  +  sf^R^xjp  =  ||R^x|l^  for  all  x  in  H2.  I  +  S  is  obviously  strictly  positive 

on  its  domain  D(S).  To  prove  that  its  smallest  limit  point  is  strictly 

1. 

positive,  one  notes  that  the  spectrum  of  (I  +  S)-'  is  bounded  below  by 

,2 


la  L  1 


inf - f  ■■  ♦  and 


rV 


iiiy-’ 


1  9 

^Xll^ 


(I+V)’'^R^^x 


rrv] 


which  is  strictly  positive,  since  V  is  bounded.  This  proves  (1)  -  (3) 
when  C^^(P)  is  finite;  the  converse  is  clear. 


Remark  1:  Suppose  that  H2  =  L^[0,T]  and  that  the  set  of  admissible  (A,  n^) 
consists  of  all  such  that  A(X)  is  absolutely  continuous  with  L^CO,!] 

,T  •  2 

derivative  a.e.  and  E  Jq  [A(X)]  (t)dt  <  P.  The  information  capacity 

L') 

C^,(P)  will  then  be  finite  if  and  only  if  y^^  is  such  that  range 
contains  all  equivalence  classes  in  I, ^[0,1]  that  are  generated  by  abso¬ 
lutely  continuous  functions  with  L-,fO,T]  derivative. 

It  can  often  be  assumed  that  the  operator  S  is  bounded,  from 

physical  considerations.  That  is,  S  will  be  bounded  if  and  only  if 
u  y 

range(R^)  c  range In  jamming  applications,  N  may  have  the  form 
N  =  J  +  V/,  where  W  is  the  original  channel  noise  and  J  is  a  jamming  noise 
independent  of  the  ambient  noise  W.  Since  W  will  typically  include  wide¬ 
band  receiver  noise,  it  is  not  plausible  that  the  sample  functions  of  the 
jamming  noise  J  should  be  more  irregular  than  those  of  the  ambient  noise 
K  The  path  properties  of  N  and  W  are  determined  by  the  properties  of 
the  RKHS  of  N  and  IV  (see,  e.g.,  flOJ).  Thus,  if  the  paths  of  W  +  J  are 
not  to  be  more  irregular  than  those  of  W,  then  it  is  necessary  that 
range (Rj^)  c  range (R^).  These  statements,  which  can  be  rigorously  justified, 
imply  that  one  can  often  assume  S  to  be  bounded.  However,  it  is  desirable 
to  state  the  results  here  in  maximum  generality,  so  S  will  not  be  assumed 
to  be  bounded. 

When  il^  is  infinite-dimensional,  n  will  denote  thi'  sriia  I  lest  liiiiii 
point  of  the  spectrum  of  S,  the  operator  defined  by  (A-2).  The  limit 
points  of  the  spectrum  of  S  consist  of  all  eigenvalues  of  infinite 
multiplicity,  limit  points  of  distinct  eigenvalues,  or  points  of  the 
continuous  spectrum  [11].  A  key  consequence  of  9  being  a  limit  point  is 
that  there  is  a  sequence  of  o.n.  elements  ff^)  such  that  jj  (S  -  01)  f^^ l|  >  0 


V.-V. 


I--.- 


[11,  p.  364].  Prom  Proposition  1,  1 +  6  >  0.  Moreover,  a  real  number  C 
with  0  <  C  <  1  +  0  can  be  in  the  spectrum  of  I  +  S  if  and  only  if  C  is  an 
'.'igcnvalue  of  finite  multiplicity  for  1  *  S.  Thus,  is  the  only  possible 


a  “  ^  a  '  ^  '  a*  .  *  ^  • 


;•  /v-v'-v-v 

Awls' .J^.» .flit  ^ 


i 
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limit  point  of  the  eigenvalues  of  S  strictly  less  than  0.  {X^,  n>.l}  will 

denote  the  eigenvectors  of  S  that  are  strictly  less  than  0;  of  course,  this 
set  can  be  empty.  Similarly,  te^,  n>l}  will  always  denote  an  o.n.  set  of 

eigenvalues  of  S  corresponding  to  the  eigenvalues  {X^,  n>l}:  Se^^  =  n>l. 

The  case  0  =  «>  requires  special  treatment.  It  is  simplified  by  the 
following  result. 

Proposition  2:  Suppose  that  0  =  <».  Then  I  +  V  must  be  compact,  and 

K 

{X^,  n>.l}  is  an  infinite  set.  Iforeover,  P  +  ^  X^  <  KXj^  for  some  finite  K, 
any  fixed  P  >  0. 

Proof :  0  =  00  implies  that  zero  is  the  only  limit  point  of  the  spectrum  of 

I  +  V,  so  that  I+V  is  conpact.  Since  I+V  is  self-adjoint,  this  operator 
has  a  c.o.n.  set  of  eigenvectors.  I+V  is  strictly  positive,  so  that  its 
eigenvalues  are  n>l},  with  Xj^>-1. 

K 

To  see  that  I  X^  +  P  <  KXj^  for  some  finite  K,  suppose  not.  Then 
K 

^  J::  ~  for  all  K^l.  This  cannot  hold,  since 

^  □ 

CtiK'  can  now  formulate  the  capacity  problem  in  terms  of  the  operator  S, 

as  follows. 

is  the  supremum  of  (2)  subject  to  the  constraint  (3’ I.  Rewriting 
(5')  in  terms  of  S,  and  using  (1),  one  obtains  the  equivalent  constraint 

-  p.  (31 

Setting  =  Tj^||  (I+S)'^U*u^|| 

q^,(P)  =  sup(4)  log  [l^X^d 

where  the  supremum  is  over  all  sequences  fX^)  and  c.o.n.  sets  (v^^,  ns  11 
in  the  domain  DfS)  of  S  such  that  f »  where  y  =  n^J. 


11 


When  S  is  bounded  and  compact,  the  results  given  here  were  presented 
at  the  1983  IEEE  Symposium  on  Information  Theory  (St.  Jovite,  Quebec, 

Canada)  and  are  partially  contained  in  [2].  .^n  upper  bound  for  the  capacity 

when  S  is  bounded  but  not  compact  has  been  given  by  Yanagi  [12], 


Capacity  for  Finite-Dimensional  Signal  Space 

Proofs  of  the  following  two  lemmas  are  given  in  the  Appendix. 


Lemma  1:  Let  ,  n£M,  be  any  non -decreasing  sequence  of  strictly  positive 
real  numbers.  Let  (X^^)  be  any  sequence  of  M  real  numbers.  Fix  P>0  and 


define 

ium.p,y)  = 


sup  il  (Yr/X^)/Y  . 
j  ^  n  n  n 

L  X„<  P} 

'  1 


Tiien 


K  K 

g(M,P,Y)  =  n  (Z  Y.  +P)/(KyJ 
n=l  1  ^  " 


whci 


•e  K<.M  is  the  largest  integer  such  that  l  y.  +  P  >  Ky,..  g(M,P,Y)  is 

1  1  -  K 


uniquely  attained  by  (X^)  such  that 


lx;  -  a  Ti  *  P)/K  -  n<K 

!  i- 


I  =  0 


n  >  K. 


Lcitmia  2 :  Let  (A^),  ^  non-decreasing  sequence  of  strict  1\  positive 

real  numbers  and  fix  P>0,  Define  a  sequence  (Yj^)  to  be  admissible  if  it  is 
J  J  K 

non-decreasing,  '.’ly-  >  lx.  for  all  j£K,  and  Zy-  '•■P  ^  Xy^-.  Define 


12 


K  K 

Fl'Cy)  =  n  (P  Sy;  )/(Ky  ) .  Tlien,  for  any  admissible  sequence  (y  ), 
'  n=l  1  ^  n 


fj^(Y)  1  equality  if  and  only  if  Yj^  =  for  all  i^K. 


Corollary  1:  Let  (v^)  and  (y^),  i  =  1, — ,M  be  two  non-decreasing  sequences 
of  strictly  positive  real  numbers.  Fix  P>  0,  and  let  K  be  the  largest 

K  2 

integer  <  M  such  that  ):y^  +  P  >  KYj^.  Let  (X^) ,  n  =  1 . M,  be  any  sequence 

M  2  ^  J  J 

such  that  I  X  <  P.  If  I  y-  >  ^  v-  for  all  J <  K,  then 

1  “  n=l  ^  "  i=l  ^ 

K  r  K 

<Z  log  (P+  Z  v.)/Kv„) 
n=l  L  i=l 

with  equality  if  and  only  if  Y  =  v  for  n<K  and 

n  n  ~ 

9  K 


f  2-1 

log  l»Vn 


=  0  n  >K. 

M  2  -1  F 

Pt'oof '•  If  log  (1  +  X  Yn  )  >  Z  log  (P  +  Zv. )/Kv  )  ,  then  it  must  also 
n=l  n=]  L  1 

K  K 

strictly  exceed  (by  Lemma  2)  Z  log  (P  +  Zy-)/(Ky  )  .  This  contradicts 

n=l  [.  1  ^  ^ 

Lemma  1.  The  conditions  for  equality  follow  from  lemmas  1  and  2. 

n 

l^cnuark  2;  If  S  is  a  bounded  linear  operator  with  a  complete  set  of  eigenvectors 

K  K 

and  non-decreasing  eigenvalues  B,  then  Z3„  <  Z  <Sv  ,v  >  for  any  o.n. 

i  ^  ri  2^  n  n 

set  Vp  . . .  ,Vj^  and  any  K  >  1  [  13] . 


Theorem  1 : 


Suppose  that  H2  has  dimension  M<-».  The  capacity  is  then 


K 


K 


C^(P)  =  (i)  ■  log 
n=l 


^  i  -i .  +  P  +  K 
1  ^ 


K(1  +  3^) 


where  Si  i  1  •  ■  •  3.,,  are  tJie  eigenvalues  of  S,  and  K  is  the  largest  integer  ^  >] 


'1-  ^2- 
K 


such  that  IS|  +  P2.KSj^.  Tlie  capacity  is  attained  by  a  Gaussian 


with 


covariance  operator  (1),  where  u^  =  Ug^  and 


IB.  +  P  -  KB„ 
,  1  n 


(l  +  3j^)'^K'^  for 


n^K,  =  0  for  n  >  K,  and  {g^^,  n^l}  are  o.n.  eigenvectors  of  S  corresponding 


to  the  eigenvalues  (3^^)  •  No  other  Gaussian  can  attain  capacity.  The 


same  result  is  obtained  if  H.,  has  dimension  L<~  and  is  constrained  to 


aa\c  .-^uppoft  of  dimension  '!<  L. 


i’noof :  Since  H2  is  finite-dimensional ,  the  self-adjoint  operator  S  is 
bounded  and  has  a  complete  set  of  eigenvectors.  From  (4), 


M 


..2  -1 


c:^,.(P)  =  sLipil  I  log  +  ]j,  where  Yj^  =  l  +  <SVj^,  v^>,  {v^^,  n<M}  is  a 

1  1  I 


c.o.n.  set,  and  tlie  supremum  is  over  all  such  c.o.n.  sets  and  all  (.V)  such 


M  . 


that  I  £  f’ •  Since  62^£02£  —  £  non -dec reusing  eigenvalues  of 


J 


S,  bv  Remark  2,  I  [l  +  <Sv  ,  v  >]  >  I  [1  +  B  1  for  all  J<M  and  any  fixed  c.o.n. 
*  '  ’  j  ‘  n’  n  —  j  n  -  ^ 


set  n<.M}.  The  expression  for  C^^.(P)  and  the  unique  covariance  of  the 


maximizing  Gaussian  both  now  follow  from  Corollar>’  1. 


n 


Remark  3;  The  result  holds  if  dimfH2)  =  L  <  °°  and  dim[supp(p^^^.^)  1  £  M  £  L, 
since  in  this  case  S  again  has  M  smallest  eigenvalues. 


T1  ICO  rein  2: 

Suppose  that  is  infinite-dimensional, and  support 

A  (Xj 

restricted  to  have  dimension  <  M  <  <». 

(a)  If  n>l}  is  empty,  then  C^^,(P)  *  (M/2)  log  [  1  +  PM'^  (1  +  0) '  V 

Capacity  can  be  attained  if  and  only  if  S  has  0  as  an  eigenvalue  of  multipli¬ 
city  ^  M.  In  this  case  is  attained  by  a  Gaussian  vvlth  covariance  (1), 

where  u^  =  Ug^  and  =  PM  ^(1  +  9)  *  for  i^M  with  {gp...,gj^j)  any  o.n.  set 
in  the  null  space  of  S-0I. 

K 

(b)  If  KAj^  ^  I'A^  +  P  <  KAj^^^  for  some  K<  M,  then  the  capacity  is  as  in 

Hieorem  I,  with  i-  =  and  can  be  similarly  attained. 

(c)  l-et  K  =  min(L,M),  where  L  >  1  is  the  number  of  eigenvalues  (A  )-of  S  whose 

—  n 


value  is  strictly  less  than  9,  and  suppose  that  P  +  XA.  KA.,.  'Hie  capacity  is 

1  ^  ~  K 

tiicn 


K 


K 

r  1 

P+  E  (A.-0) 

1  +  0 

4- 

log 

1  +  i=l  " 

- 

l2j 

M(l+0) 

C,.(P)  =  1  Z  log 
n=l 

The  capacity  can  be  attained  if  and  only  if  9  is  an  eigenvalue  of  S 
with  multiplicity  >  M-K.  The  capacity  is  then  achieved  bv  a  Gaussian  y.,.  with 

—  '  /\A 

^  -1  -1 

covariance  (1),  where  u  =  Ug  and  x  =  (EA.  +  P  -  MA  +  (M-K)9)(l  +  A  )  M  for  n<K, 
’  n  "^n  n  ^  1  n  ^  n^  ’ 

with  Sg_  =  A  g^  and  {g,,...,g..}  an  o.n.  set;  and  with  u  =  Uv  and 
Ti  n^i  JL  K  n  n 

^  -1  -1 

Tn  =  (P  +  EA^  -  K9)M  (1  +  9)  for  K  +  1  ^  n  ^  M,  where  Sv^^  =  and  ,  •  •  • 
is  an  o.n.  set.  The  sets  {u^^, —  ,Uj^}  and  {xj^ . Xj^}  are  uniquely  defined  for 


any  iiui.ximizing  (iiiussian 


Proof :  If  S  >  61,  then  S  -  01  does  not  have  zero  as  an  eigenvalue. 

However,  there  exist  [ll]  o.n.  elements  (fj^)in  D(S)  such  that  H  (S-6I)fj^jl  -*•  0, 
so  that  ''(S-0I)f  ,f  '  *•  0,  <Sf  ,f„''  6,  <Sf  ,f  >  '  6  for  evcrv  n.  Thus,  for 

any  t  0  there  are  o.n.  elements  f j ,f^j  such  that  setting  =  ■‘SfV,f9>, 

6  '  Yj^  <  0  +  !_  for  i_<M.  Using  this  sequence  in  (2)  and  (4),  one  obtains 
M 

=  (1)  Z  log  [1  + 

=  ci)  X  log  (1 +Y^)  ']  1  (1)  log  [1  +  x;;  d  +  e  +  ei  ']• 

2 

The  expression  on  the  right  of  the  inequality  is  maximized,  over  all  fX^)  such 

M  7  M  .1 

that  Y  Xj^  <  P,  by  defining  =  PAf,  n  ^M.  Thus,  (^(P)  >  log  [1  +  (1 

M 

* ‘  -1  -1 

for  all  e  >  0,  and  so  Cj^CP)  ^  (*5)  T  log  [1  +  M  Cl  +  0)  ]•  For  the  reverse 

inequality,  one  notes  that  under  the  constraint  F.  ||R^^ACX)|  <  P»  it  is 

UX  N  - 

shown  in  [1]  that  Cj^^CPl  =  (M/2)  log  (1  +  P/M) .  For  S^ei, 


11^  TV>nc  P  IIR*^  AfYl  11^ 


j|l^‘^\(X)  il^  ::(l  +  9)'^  !|R^'^A(X)|!^.  Thus,  ||R^^A(X)ir<  P  implies 

X 

F,^!|R^‘^A(X)1|^  <  C1  +  0)‘^P,  giving  C,^(P)  <  f^j  log  [1  +  ,  so  that 

q^.(P)  =  (M/2)  log  [l  +  PM‘^(l  +  e)'^]. 

If  Sy0I,  with  9  an  eigenvalue  of  multiplicity  K,  the  above  argument  is 
modified  in  an  obvious  way  (Yj  =  0  for  i=l,...,min  (K,M))  to  again  obtain 

=  (M/2)  log  [1  +  PM'^(1  +  9,)'M. 

lo  prove  (bj,  the  proof  of  'Iheureni  1  is  repeated  after  substituting 
for  .-Y,  i;_K+l. 

ioi  lO),  suppose  that  S  has  K  M  eigenvalues  A^  <  ...  <  Aj.  strictly  less  than  0 


and  that  YA.  +P  >  KA^..  C^,iP)  =  sup  f-wiPi  .v)  where 

1  "  “■  ^  ff’i.yi 


C^,(Pl,v)  =  sup  I5  I  log  [1  +  X^  (l  +  <Sv^,  Vj^>2)  ] 

Ln=l 

2 

V  =  {Vj^,  n^M}  is  any  o.n.  set,  0<Pj^<P,  and  the  siqiremum  is  over  all  CX^) 
K  ^  M  . 

such  that  E  X“  <  P, ,  E  X  -  P.  From  the  prools  of  Theorem  1  and  part  (a), 
1  "  ^  1  ^ 


=  ih)  L  log 
n=l 


T,*p 

i 


"  V  a 


+  (J5)(^^K)  loj: 


P  -  P, 


1  + 


fM-K)Ci+e) 


j 

where  J<K  is  the  largest  integer  such  that  ZA.  +P,  >  JA..  Since  this  result 

^  i  1  —  %j 

holds  for  any  o.n.  set  {v^,  n<M}  in  D(S),  it  remains  only  to  determine  the  value 
of  P^  that  maximizes  CjyCPj^jV^)  (a  differentiable  function  of  P^^  in  [0,P]).  Differ 
entiating,  one  sees  that  is  increasing  with  Pj  so  long  as 


P,  ^  fJP  +  Ol-KUde  -  ZA.)]ffl-K+J)‘^.  Since  P 
^  1  ^ 

inequality  is  <atisfiel  as  long  as  C?i-K+J)Aj^j 

J 


J 

<  the  preceding 

J 

-  ZA.  <  P  +  (M-K)9  and  this  is 
1  ^ 


satisfied  because  P  +  'A^  X  d^j+l»  <  S-  It  follows  that  C,y(Pj,v)  is  an 

K  K 

increasing  function  of  P^  for  P^  <  -ZA^  +  KAj^.  Assuming  that  P^  >  -ZA^  +  KAj^, 


the  maximum  of  (^^(P^.v)  is  attained  uniquely  by  P,  =  M  [KP  -  Ot-K)ZA.  +  (II-K)KO]. 

1  ^ 

Using  this  value  of  P^  in  the  expression  for  C^yCP^^.v),  one  obtains  C^.,CP)  as  in 
(c).  The  value  of  C^^CP)  when  L  =  M  in  (c)  follows  as  in  the  proof  of 

fheorem  1,  Tne  statement  on  attaining  capacity  follows  from  the  results 

of  (aj  and  (b) . 


1 


Corollary  2:  If  0  =  «>,  then  Cj^CP)  has  the  value  given  in  Theorem  2(b), 
and  can  be  similarly  attained. 


Proof:  Follows  from  Proposition  2  and  the  proof  of  Theorem  1. 


Capacity  for  Infinite -Dimensional  Signal  Space 


Theorems  I  and  2  give  the  solution  to  the  capacity  problem  when  the 
dimension  of  tlie  signal  space  is  finite.  We  now  proceed  to  the  case  of  an 
infinite-dimensional  signal  space. 


Lemma  5:  Suppose  that  9<oo,  (X^,  n>l}  is  an  infinite  set,  and  P  >  0.  Then 
K 

P+  >;A.  >  K\,.  for  all  K  >  1  if  and  only  if  P  >  Z  (6  -  X  ), 

^  ~  ■n>l  " 


Proof:  It  suffices  to  show  that  Z(X  -  6)  +  P  >  K(X„  -  6)  for  all  K>  1  implies 
-  2^  n  —  K  — 

1  [  '  -  P.  Suppose  not.  Then  there  exists  K>  1  and  A>  0  such  that 

K  K 

Thus  [e  -  ^  6)  -  P]/K  =  A/K.  Suppose 

K+N 

i  ■  K+p'  -  I''  ■  ^k+N+1^  -  ^^n‘ 

K+N 

=  [A-  ::  (X  -  e)]/(K+N)  [A  +  NA/K]/(K+N)  =  A/K.  TIius,  the  induction  hypothesis 
K+1 

would  yield  that  [0-  ^  A/K  for  ail  N^i.  However,  9  is  the  smallest 

limit  point  of  the  spectrum  of  S,  and  since  (A^)  is  a  bounded  infinite  sequence, 

must  contain  a  limit  point.  Thus,  9  -  Aj,_|_j^.  ^  A/K  for  all  N^l  would  mean 

that  (Aj^)  kis  a  limit  point  strictly  less  than  0.  This  contradiction  implies 
K 

that  P  +  Z(A^  -  0)  =  -A<  0  must  be  false. 


Lemma  4:  Suppose  that  9  <  oo  and  S  -  91  is  negative-definite  with  an  infinite 


set  of  strictly  negative  eigenvalues 


1...D  [P+  ^CA  -0)' 

(a)  If  P  >  Z(0-A  ),  then  q^(P)  =  I5  z  log  - 
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The  caoacity  can  be  attained  if  and  only  if  P  =  rC6-A^).  It  is  then 

n 

attained  by  a  Gaussian  with  covariance  operator  (1 ) ,  where 
and  ^  n>l. 


where  K  <  <»  is  the 
K 

largest  integer  such  that  P  +  _>  KX,^.  The  capacity  can  be  attained 

by  a  unique  Gaussian  with  covariance  operator  fl),  where  u^ = Ue^  and 
K 

EX^  +  P  +  K 

"n '  Wrr  ■  ^ 

n 

=  0  for  n  >  K. 


K 

(b)  If  P  <  E(0-XJ,  then  C^^(P)  =  I5  E  log 
n  n=l 


EX.  +  P  +  K 
1 


Tcrfy 


roof:  (a) •  The  fact  that 


2C„(P)  >  E  log  [^] 
n>l  *•  n** 


P+  i„(x„-e) 
mm 


1+0 

follows  from  (c)  of  Tlieorem  2,  letting  M  +  ^  in  that  result.  To 
prove  the  reverse  inequality,  suppose  that  C^^,(P)  is  strictly  greater 
than  its  value  as  given  in  Ca) .  Then  for  e  in  C0»C^^CP))  there  exists 


a  Gaussian  with  covariance  =  ET^[Rj^^]  ®  where  {u*^,  n'll 

n  ‘ 

is  a  c.o.n.  set,  all  x*' >  0,  Ex^  ||  CI'*'S)U*u  "||  <  P,  and 

n  n'  n" 

n 


Cy^CP)  =  4  ^  log  (1  +  xh  +  E.  Since  E  log  (1  +  is  non-decreasing  with 
n  1 

M,  for  some  c  >  0  there  must  exist  <  «>  such  that 


log  fl  +  T_1 


1  P+  E  (A  -0) 

*  t!i  >  .  r  loo  ii?-  * - . 


Ihe  UlS  ol  (.j)  is  the  value  ol'  llie  inroiiiial  ion  when  emai  iaiu  e 


operator 


A(Xj  n  N  n  N  n 

From  Theorem  2(c),  the  LHS  (5)  can  be  no  greater  than 

1  +  P+  y  (A.-0) 

‘‘f  '“S  *  T  “>8  - . 

1  [  nj  M(l  +  e) 

As  -*•  this  last  expression  converges  upward  to  RHS  (5).  Thus,  the 
inequality  (5J  cannot  hold. 

To  see  that  the  capacity  is  attained  as  stated  in  (a),  one  notes  that 

from  (1)  the  Gaussian  measure  with  covariance  (1)  will  achieve  capacity 
if  and  only  if 

O  r,  ri*6  1  .  P* 


r  log  (l*g  -  I  log  [^1  ^ 

n  ^  n'^ 


rTn(l+>^)^P,  (7) 

the  latter  requirement  following  from  the  definition  of  (X^) ,  Corollary  1 

and  Remark  1.  Both  (5)  and  (7)  are  satisfied  if  x  =  (6-A  )(1+A  )"^  for 

n  n  ^  n 

all  n>l  and  Ef9->^)  =  P.  Conversely,  if  (x^)  satisfies  (6),  then 


P+  X  (A  -0) 

Tl  e  '’n*‘n‘'n'n O'-  ^  1  *  V  • 

n  n 

with  equality  if  and  only  ifx+X+xA  =  fl  for  all  n  >  1.  If  (x  ) 

nnnn  —  n 

also  satisi ics  (7),  then  necessarily  P  =  (1  +  A  ),  and  so 

n  n 

=  0  for  all  n>l.  Thus,  if  (r^)  satisfies  both  (6)  and 
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K 

c^(P)  "  4  r  log 
~  n=l 


fK  1 

i=l  "  I 

[-TTuy- j 


18) 


follows  from  Cb)  of  Theoi'cni  2.  Suppose  that  C^.rfP)  >  RliS  (8),  Then 
by  (4)  there  exists  a  c.o.n.  set  (v^^,  n>l}  and  a  sequence  with 

infinite  number  of  non- zero  terms  (using  (b)  of  Ibeorem  2)  such  that 

RHS  (8)  <  Is  I  log  [1  +  X^(1+ <SVj^,v^>  )'^]  (9) 

n 

-> 


with  2X“  <  P.  Since  rilS(9)  is  finite  and  the  sun  of  non-negative 
n  ^ 

terns,  there  must  exist  M  <  «.  such  that 

M  2  - 1 

RHS  (8)  <  ^  Z  log  [1  +  X  (l  +  <Sv  ,v  >  )  ].  This  contradicts  (b) 

n=l  ”  ^ 


of  Theorem  2. 

n 

Lemma  5:  If  a  < «  and  S^9l,  then  Cj^(P)  =  2-'^^Vq^  • 

Proof:  C^yCPl  >  PC1  +  9)  ^2  ^  follows  from  part  (a)  of  Iheorem  2  by  letting 
M  -*■  To  prove  the  reverse  inequality,  one  notes  that  for  the  constraint 

A(X)  II "  <  (1  + -•)  ^P,  the  capacity  C^([l+6]  ^P)  is  P(l+0)  ^2  ^ 

X 

[1,  Theorem  2].  Since  E  !|R.'^  ACX)  ir  <  P  implies  R  ||lC''^  A(X)  ||^  <  (1+  e)'^P, 

^  ^x 

optimization  w.r.t.  the  former  constraint  is  over  a  smaller  set  than  w.r.t.  the 
latter  constraint;  thus  C^^CP)  £(^[(1  +  9)  ^P]. 

Tlic  capacity  for  channels  with  an  infinite-dimensional  signal  space 
can  now  be  given. 


Theorem  .S:  Suppose  that  9  < »,  H2  is  infinite-dimensional,  and  dim[suppfii^.^)  I 
is  not  constrained. 

(al  If  (A^,n_>l}  is  not  empty,  and  ^  P,  then 


log 


ri+'- 

i+>. 


p  + 


y  (\  -n) 
m  m 


n- 


1  +  0 
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(b)  If  {X^,  n>l}  is  not  entity,  and  P  <  then  there  exists  a 

largest  integer  K  such  that 

p 

(c)  If  {X^,  n>l}  is  empty,  then  C^(P)  =  • 

(d)  In  Ca) »  the  capacity  can  be  attained  if  and  only  if 

It  is  then  attained  by  a  Gaussian  with  covariance  operator  as  in  (I), 

where  u  =  Ue  and  x  =  (G-X  )(1+X  )  ^  for  all  n  >1.  In  (b) ,  the  canacity 
n  n  n  n  n  • 


ZXi  +  P  ^  KXj,,  and  C^CP) 


^  j:x.+p  +  k 

K 

h  ^  log  K(l+  X'  1 
n=l  ^  n 


can  be  attained  by  a  unique  Gaussian  y.y  with  covariance  operator  (1), 


EX.  +  P+  K 

where  u  =  Ue  and  x  =  — v?/, .  - 1  for  n<K:  x  =0  for  n>K.  In 

n  n  n  KCl+X^J  —  n 


Cc) ,  the  capacity  cannot  be  attained. 


Proof :  From  the  preceding,  one  must  find  sup  log  [1+  x^],  where  (x^^) 

is  a  non-negative  summable  sequence,  subject  to  the  constraint 

E  x„  [1  +  <USU*u  ,u  >]  <  P,  where  {u  ,  n>l}  is  any  c.o.n.  set  in  and 
n  n  n  n  —  n  —  '  L 

=  R^yCl  +  Sj'^U*,  with  U  unitary. 

Let  Q  be  the  projection  operator  onto  the  (closed  linear)  subspace  spanned 
by  {Ue^,  n^l},  where  {e^,  n>^l}  are  eigenvectors  of  S  -  9l  corresponding  to 

strictly  negative  eigenvalues.  Let  be  the  projection  onto  the  orthogonal 
complement  of  range(Q)  Rangc(Q)  is  obviously  an  invariant  subspacc  for 
U(S-  '  IJIP  and  thus  for  USU*;  since  USU*  is  self-ad  Joint ,  rangefQ'*')  is  also 
invariant  for  USU*. 
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f.  ■-  ■ 

r.‘-- ; 

f.  ■ 


The  set  of  covariance  operators  T  =  satisfying  the  above  constraint 

consists  of  those  T  such  that  Trace  T^[Q +  QUSU*Q]T^  =  and 

Trace  T'^[Q-^  +  Q-^USuV]T^  ^  P-P;^,  where  =  P^CT)  is  contained  in  [0,P]. 

Thus,  the  capacity  problem  is  to  determine  sup  sup  Trace  log  (I +  T) , 

P,  in  [0,P]  A,nk-y 

where  ^  ^  ^ 


Aj  =  ^fP^)  “  (covariance  operators  T:  Tr  T^CI  +  bSU*)QT'^  “  ^1^ 

and 

operators  T:  Tr  T'^^(I  +  USII*)qV'^  <  P-Pj^}. 

Now,  sup  sup  Tr  log  (I  +  T) 

Pj  in  [0,P]  Aj^a\2 

<  sup  (sup  Tr  log  (I+T, )  +  sup  Tr  log  CI  +  T,)!  . 

P^  in  [0,P]  \  A^  ^2  / 


it  is  shoun  below  that  this  inequality  is  actually  an  equality. 


Proof  of  (a).  First  suppose  that  (X^,  n^ll  is  an  infinite  set,  and  fix 
P^  in  [0,P].  Suppose  that  Z(Q-X^)  >  P^^.  By  Lemma  3,  there  exists  a  largest 

K 

integer  K  such  that  IX  +  P  >  Kx^r.  Cup  %  Trace  log  (I  +  T)  is  then  (from  part 

1  ""  -  Aj(P^) 


K 

fb)  of  Lemma  4)  I5  z  log 
n=l 


^  I 

I  X.  +  p,  +  k! 

i=i  1  ^ 

“TTrry 


.  From  Lemma  5, 


sup  Trace  log  (I  +  T)  =  %  CP-P,)(l+9)  Thus,  for  this  value  of  P, , 
A.(P,)  ^  1 


sup  c  Trace  log  (I  +  T)  <  C^(P.,K)  "  Z  log 
A^nA,  n=l 


Z  X .  +  P^  +  K 
i=l  ^  ^ 


Tcrmgr  j 


*  !iCP-Pj)(i+e)'L 


1 


c^Cp^.K)  = 


YCi  +  0)  whose  sign  is  determined  by  that 


EX.  +  P,  +  K 
1  ^  ^ 


K  K  K 

of  Kv  -  P  -  since  .  +  P  +K  >  0  O-ecause  E> .  +  P^  -  Ka„  and  '  >  -1). 

1'*'^  ^li~K  K 


<  KXj.^^  <  K6,  so  is  increasing  for  P^^  increasing  wh 


K'.^.  .  Pj  *  ::Xj  <  .  Defme  P  -  -  r  a  ^ 

i  1=1 


r-  z'  r\  ^  I ' 


2CJP-.K1  =  I  log  — 


l.X  1 


1  1  +  X 

n=l  n 


.  Using  the  inequalities 


^■^+2  —  ^K+1  ^  *  ^K+'’  C_j_(P  ,K)  is  seen  to  be  a  strictly  increasing 

function  of  K,  Since  (•  -  •  1  P,  K(e  -  A,,)  ->  0,  and  thus 

n  —  K 

.  r  K+i 

liin  P,  =  lim  E  (A.  -  aj  +  -  o)  =  E((}-  Xj. 


K  '  K  Li=l  ^  J  1  ^ 

This  gives  as  an  upper  bound  for  the  capacity,  for  all  P^^  such  that  P^^  <  E(a  -  x^) , 

n 


K  n  +  a1  [P  -  E(6  -  X  ) 

in,  C,(P^K)  .  4  1  leg  *  4 


Since  P  ->  r(''-  x^) ,  there  exist  P^  values  satisfying  the  constraint 
with  Pj^  i(';  -  x^3 .  In  this  case,  from  part  (a)  of  Lemma  4, 

r-  [P,  +  E(x  -  all  P  -  Pn 

ffp  ■)  =1-  r  ,  j  ^  ^  +  l,  _! _ D _  +  _ 1. 

j)  -2  ^  log  1+;^  +  'S  I  1+0  M  +f, 

n  +  o'i  IP^ 


.  ..  ,  n  +  Gl  .  .  I'  "^n^'n 

=  ^  ^'n  — rrr 


ihe  last  expression  is  thus  an  upper  bound  on  C^^-CP)  for  all  P^  in  [0,P] 


From  part  (a)  of  Lemma  4,  this  is  the  value  of  the  capacity  when  the  operator 

S  '  0l  is  negative  definite  with  an  infinite  set  of  strictly 

negative  eigenvalues.  Thus,  this  is  the  capacity  when  =  P,  and  so  (a)  is 

proved  if  {.Xj^,n>l}  is  an  infinite  set. 

Suppose  next  that  the  set  {x^,n>l}  is  finite,  Xj  <  X2  s  ...  <  X^,  <  G. 

K 

Proceeding  as  above,  for  Pj^  such  that  ^  ^  upper  bound  CCPj,J) 

1  K 

on  the  capacity  is  increased  by  increasing  Pj  up  to  the  value  P^  =  Kx^,-  J:  x^. 

y  1 

If  +  ):  Xj^  >  KXj^,  then  from  above, 

K  (P-P) 

r»  _  1  r-  1  -LX  ,  t 


cCPj.K)  -  ij  E  log  — ktjttI —  ~T 

n=l  n-^ 


Differentiating  w.r.t.  P^^,  one  sees  that  the  derivative  is  positive  for 


■  x.  P  -  K'''  •'  0, 

ill  ’ 


xv. 

negative  for  EX  -  +P,  -  Kn  >  0,  and  so  the  unique  maximum 
1  ^  ^ 


occurs  for  P,  =  );(g  -  x.).  This  gives  an  upper  bound  on  the  capacity  of 
^  1  ^ 

K 

K  ll.nl  P  +  -  e) 


1  +  X. 


+  b  1 


1  +  G 


To  show  that  this  upper  bound  is  actually  the  capacity,  a  sequence  (rh)  of 

covariance  operators  will  be  exhibited,  each  satisfying  the  constraint,  and 

such  that  sun  b  trace  log  (I  +  T(,)  is  equal  to  the  unner  bound.  Thus,  fix 
r,-D‘,M>K 

M  w 

^  ''0  such  that  *  1+e,  Por  ?I>K,  define  T.,  bv  T.'i  =  E  t^’^u^®u^  where 

M  '  -M  _ ,  n  n  n 

n- 1 


for 


P+  EfX.  -  9) 
1  ^ 


1  <  n  ^  K 


K  <  n  <  M 


Set  =  IJe^,  n=l, ,K.  Choose  the  o.n.  elements  for  n  =  K+l ,  —  ,M 

such  that  <USU*u^,  u^>  <  9  +  c;  this  is  possible  for  any  M,  because  6  is 
the  siiallest  limit  point  of  the  spectrum  of  S.  One  now  obtains 


M 


E  T^’^Vl  +  <USII*u"-,u^>  ]  <  E  CO  -  A^)  +  E  +  0  +  c]  =  P. 

n=l  ^  n  n  -  n  n 


K 


M 


Moreover 

K 

is  Trace  log  (I  +  T»jl  =  ^  E  log 

n=l 


1  +  0 


1+A 


•¥log 


IT" 


K 

P  +  E  (A.  -  0) 

1  +  i=l 

^  CM-K)(1+  0  +  e) 


For  fixed  e,  the  limit  as  M^-a>  of  thisexpression  is 


K 

P+  E(A.  -  e) 
+  1  ^ 


2(1+  0  +  e) 

Since  e  >  0  is  arbitrary,  one  sees  that  the  sunremum  over  all  c>0,  M>K  is 
equal  to  the  upper  bound  previously  obtained.  That  upper  bound  is  thus  the 
capacity  Cj^CP),  completing  the  proof  of  (a). 

The  result  of  (b)  can  be  obtained  from  the  proof  of  (a).  Since  now 


E(9-  A^)  >  P,  there  exists  a  largest  integer  K  such  that  P+  EA^^  ^  KAj^,  by 


n 


Lemma  3.  Choose  Pj^_^P  and  proceed  as  in  the  proof  of  (a)  to  obtain  an  upper 
bound  on  the  capacity  of 


M 

CCP,,M)  =  is  E  log 
^  n=l 


•  M 

E  A.  +  M  +  P, 
i=l  ^  ^ 


irrmrj 


n' 


+  ^ 


(P  -  P^) 
1  +  0 


M 


where  M  is  the  largest  integer  (note  M^K)  such  that  Pj^  +  EA^^  ^  Defining 

P^*  as  in  the  proof  of  (a),  the  sequence  (CCP^.M])  is  non -decreasing  as  M 
increases,  and  since  K  is  the  largest  integer  such  that  P+EA„  >  KA,.,  one  has 

•  n  —  l\ 


1 


that  CfP,K]  is  an  upper  bound  on  the  capacity.  This  is  the  value  of  the 


capacity,  from  (b)  of  Lemma  4,  when  S  -  0l  is  negative  definite  (i.e.,  P  =  P|^). 
(c)  follows  from  Lemma  5. 


Th^^tatement  of  (d)  that  the  capacity  cannot  be  attained  when  S  -  Ol  is 
non-negative  definite  (part  (c))  can  be  proved  by  noting  that 

r!"  =  RjjfS  -  61  *  Thus  l|x||^=.  llR^Vll^ 

£  CHS  -  91+  (1  +  0)I||  ]| R^'^x  11“  ^  (1  +  0)  ^||x||^.  A  solution  attaining 

2 

the  capacity  P/ [2(1+6)],  subject  to  the  constraint  H  ||A(x)  _<  P,  would  thus 

2 

satisfy  the  constraint  E  ||A(x)  Hj^  £  P/(l+0) ;  this  is  unpossible,  by 

^x 

[l.  Theorem  2]. 

The  statements  in  (d)  on  attaining  the  capacity  in  parts  (a)  and  (b) 
follow  directly  fran  corresponding  statements  in  Lemma  4,  as  follows.  The  fact 
that  the  capacity  in  (a)  cannot  be  attained  when  {X^,n>l}  is  an  infinite  set 
follows  from  the  fact  that  Pj^  =  P  (in  the  proof  of  (a))  uniquely  gives  the 
capacity,  and  this  gives  the  same  result  as  when  S  -  0l  is  negative  definite. 

The  fact  that  the  capacity  cannot  then  be  attained  is  contained  in  part  (a)  of 

Lemma  4.  If  {X^,n>l}  is  a  finite  (nonempty)  set,  then  the  capacity  is  uniquely 

K 

obtained  by  setting  P  -  P,  =  P  +  Z(X  -  0),  corresoonding  to  the  constraint 

i  1  n 

Trace  T^[0^  +  (2'’’USU*Q'‘']T'^.  Since  (^■*’ +  is  non-negative  definite, 

application  of  the  result  for  part  (c)  shows  that  the  capacity  cannot  be 
attained.  Finally,  the  statements  on  attaining  capacity  in  case  (b)  follow 
directly  from  part  (b)  of  Lemma  4,  since  the  capacity  in  (b)  is  uniquely  obtained 
by  setting  P^  =  P,  equivalent  to  S  -  91  being  negative  definite. 

Corollary  3:  If  0  =  «>,  then  Cjy(P)  has  the  value  given  in  Theorem  3(b), 
and  can  be  similarly  attained. 

Proof ;  Apply  Proposition  2  and  the  proof  of  Theorem  3(b). 


Comparison  of  C^(P)  and  Cj^(P) 


For  the  finite-dimensional  channel,  the  capacity  C^^CP)  given  in  Theorem 

M 

1  is  strictly  greater  than  Cj^CP)  (=  ‘'S  log  [1  +  P/M])  if  ^  0,  or  if 

K  ^ 

1  verification  is  omitted. 

For  the  infinite -dimensional  channel,  a  general  statement  can  be  made  if 
{X^,n>l}  is  empty.  Then,  Cj^(P)  >  Cj^CP)  if  9  <  0,  C^^CP)  <  (^(P)  if  6  >  C^(P)  =  Cj^CP) 

if  9  =  0;  see  Theorem  2  (a)  and  Theorem  3  (c).  Note  that  f^CP)  =  P/2  for  the 
unconstrained  channel  [1,  Theoron  2]. 

If  {X^,n>l}  is  not  entity,  then  for  the  unconstrained  channel  the  value  of 

P  -1 

Cj^(P)  given  in  Theorem  -iCa)  is  greater  than  >  using  log  x  >  1-x.  This 

inequality  can  also  be  shown  for  the  value  given  in  Theorem  3fb),  proceeding  as 
in  the  proof  of  part  (b)  of  the  Theorem  in  [2].  Thus,  for  the  unconstrained 
channel,  (^(P")  >  Cj^CP)  if  9  ^  0  and  {X^,n>l}  is  not  enpty.  A  similar  result  can 

be  obtained  for  the  constrained  channel.  I 


A'.'.  V 


Discussion 


The  mismatched  channel  differs  from  the  matched  channel  in  several  ways. 
First,  the  value  of  the  capacity  can  be  very  different,  as  already  seen. 
Secondly,  the  problem  of  attaining  capacity  is  much  more  significant.  Even  in 
the  finite-dimensional  channel  the  vectors  u^^, . . .  ,Ujj  must  be  a  specific  set  of 
vectors,  not  just  any  o.n.  set.  If  H2  is  infinite-dimensional  with 
dim[supp(y^j.^j) ]  <  M,  the  situation  is  even  worse  in  (c)  of  Theorem  2.  Tliat 
is,  capacity  can  then  be  attained  only  if  S  has  zero  as  an  eigenvalue  of  mul¬ 
tiplicity  >  M  when  S  <  01 ,  or  of  multiplicity  >  M-K  when  S  has  K  <  M 

K 

eigenvalues  ‘  ‘  -  'k  ®  ^  ^  ^^i  ^  Otherwise,  in  order  to 

approach  capacity,  one  will  need  to  put  part  of  the  available  "energy"  P  in 
elements  (De^)  where  (e^)  are  eigenvectors  of  S  corresponding  to  successively 


smaller  eigenvalues.  In  practical  applications,  this  typically  corresnonds  to 
('igenf unctions  at  higher  and  higher  frequencies. 

For  the  infinite-dimensional  channel  without  a  constraint  on  dim[supp(u^^) ] , 
again  there  can  be  significant  differences  between  Cj^(P)  and  C^CP),  depending 
on  (ft;  n>l}.  However,  in  this  case  one  sees  a  rather  different  situation  in 
the  problem  of  attaining  capacity.  Cj^(P)  can  never  be  attained;  C^^(P)  can  be 
attained  if  and  only  if  {i^,n>l}  is  not  empty  and  P  < 

It  may  be  noted  that  the  results  given  in  Theorem  1  and  Tneorem  arc 
similar  to  those  obtained  in  [4,  p.  170],  although  the  developments  are  quite 

different.  However,  these  previous  results  are  given  in  terms  of  a  constraint  on 

2 

F.  ||A(X)  II  ,  and  assume  that  the  noise  variance  components  can  be  arranged  in 
ascending  order.  This  can  only  be  done  if  the  channel  is  finite-dimensional. 

In  that  case,  one  can  take  the  identity,  and  thereby  use  a  true  power 

constraint.  (A-Z)  then  becomes  =  I  +  S,  and  the  capacity  is  as  given 

in  Ihcorcm  1;  this  agrees  with  the  referenced  results  in  [4]. 

I 


The  results  given  here  provide  a  complete  and  general  solution  to  the 
information  capacity  problem  for  the  Gaussian  channel  without  feedback, 
so  long  as  the  constraint  can  be  given  in  terms  of  any  covariance  (or 
RKHS  norm).  Moreover,  the  formulation  of  the  problem  as  developed  here, 
and  the  availability  of  these  results,  are  already  leading  to  a  number 
of  related  results,  and  additional  applications  and  extensions  seem  likely. 

For  example,  it  is  well-known  that  feedback  does  not  increase  infor¬ 
mation  capacity  of  a  large  class  of  matched  Gaussian  channels,  including 
the  "white  noise"  (Wiener  process  as  noise)  channel  [6],  [15].  It  can  be 
shown,  using  the  results  given  here,  that  capacity  is  increased  by  feedback 
for  a  large  class  of  mismatched  channels,  thus  validating  a  long-held  conjec¬ 
ture.  In  another  direction,  as  discussed  above,  these  results  enable  one 
to  analyze  jamming  channels  when  information  capacity  is  used  as  the  cri¬ 
terion.  Other  related  results  can  also  be  obtained,  based  directly  or 
indirectly  on  the  formulation  and  results  given  here.  Examples  include 
capacity-per-unit  time  for  mismatched  channels  with  and  without  feedback, 
coding  capacity  for  various  types  of  channels,  and  new  relations  between 
optimum  filtering  and  optimum  transmission  in  the  Shannon  sense. 

The  framework  used  here  requires  that  the  signal  and  noise  sample 
functions  lie  in  a  real  separable  Hilbert  space.  This  is  easily  extended 
to  separable  Banach  spaces,  such  as  C[n,l]  (see  [1,  p.88]). 
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Proot  of  Leinnti  1:  Define  f^,:  R‘  >  R  by  f,,ty)  =  L  log  |l+y„  y_  1-  f..  is  to 

- — — —  j—  r).“l  n  n  M 

be  maximized  subject  to  the  constraints 


g(y)  =  ^  Xn  -  P  1  0 

1 

h^(z)  =  -yi  1  0.  i  =  1 . 

Th  is  is  a  constrained  optimization  problem  with  objective  function  which  is 

«  « 

strictly  concave  over  the  convex  set  {Z  in  R*' ;  Z^^O,  i  =  l,...,M}.  Moreover, 
each  constraint  function  is  linear.  Thus,  a  solution  to  this  problem  will  define  a 
unique  global  maximum  for  f.^j[14] ,  In  order  that  ^  be  this  unique  solution,  it  is 
necessar)'  and  sufficient  that  the  following  set  of  equations  be  satisfied  1141; 


+  S  -  =  0  i  =  1,...,M 

i  '  i 


X  y*  -  P  <  0,  B[i  y*  -  P]  =  0 

in-  ^  n 


-y?  ^  0,  =  0,  i  =  1,...,M 

for  some  set  of  non-positive  real  numbers  {B.Oi,. . . 

first,  attempt  to  obtain  a  solution  by  setting  =  a2  = 
This  requires  B(Yi+y^l  =  -1  for  i  =  thus, 


(a-lj 


(a-2) 


(a-3) 


M  M 
X  y?  +  z 


M  M 


X  y?  +  E  Y;  =  -MB  ,  and  so  y*  =  (Z  y*  +  E  y-)/M  -  y 
1  1  1  1  n  1  1  ^  1  n 


for  n  =  1,2,...,M.  This  definition  of  y*  and  the  constraints  (a-3)  rec^uire  that 


M  M 
E  y?  +  E  Y-  ■  My 
1  ^  1  ^  ^ 


for  n  -M;  this  inequality  is  satisfied  for  all  n<M  if  and  only  if  it  is  satisfied 

-1  M 

for  n  =  M.  Also,  B  =  -(y*-*- Yp  for  i  M  implies  B  O,  so  that  X  y|  =  P  by 


constraints  (a-2)  .iiavns,  X  ■  f%.  an  optimum  solution  is  given  bv 


y*  =  CP  +  Sy„  -  ^^Y,0/M,  ilM* 
1  1  ^  ^ 


If  there  exists  K<M  such  that  Kyj^  £  <  CK+  l)Yj(+'l^»  then 

^  -1 

constraints  (a-l)-(a“3)  are  satisfied  by  choosing  3  =  -KCP  +  IyJ  , 

1 

K 

T.y^  =  P 


V*  =  0,  i  >K 
'  1 


y?  =  K'Vp  +  |y^-Ky.].  i<K 
=  -KfP+p^]  ^  +  Y^^«  i>K- 

Thus, 

M  ,  K 

sup  IT  CYj^*X^)/Yj^  =  n  (ZYi  +  P)/(KYj,) 

M  y  n*l  n»l 

(Y:  E  <P} 

-  1 

K 

where  K<M  is  the  largest  integer  such  that  Ey^  +  P  ^  Xy^-.  The  suoremum  is 


attained  by  as  defined  above,  or  for 
2  ^ 

K  "  jYiJ/K- Yj,  n<K 

=  0  n  >  K. 


Proof  of  Lemma  2:  Note  that  fj^Cx)  increases  with  decreasing  Yj^  for  l^n^K 

K  K  K 

using  Z  Y  -  +P  >  Kyl--  >Uso,  one  can  assume  that  E  y„  =  E  y„-  To  see  this 

1  -  N  n  =  1  "  n  =  l  ” 


suppose  that  ^  Yjj  ^  "V'  admissible  sequences  (>'n  )»  jll»  3s 

n=l  n=l 

follows.  =  Yj^j  l_<n£K.  Given  (Y^‘^^)f  let  Pj  be  the  largest  integer 

m 

i^K  such  that  -  (y„  -  A  )  =0,  If  no  such  integer  exists  for  j  =1,  set 

n=l  “  ^ 

p^=0.  Define  by 


.  .U) 

n  n 


f^lPj 


=  Y^^^  -  Ej  Pj  <n<  K 


where 


t. .  =  min 


—  I  (y*-^'^  -  a  ):  p.  <  J<  k| 
•Pjn=Yl  "  ""  J 


To  see  that  is  non -decreasing,  it  suffices  to  check  that 


Tills  holds  because  e.  _<  yPL  -  noting  that 

\i  Pj  J  Pj  J-  Pj 


>  A  > 


'p.+l  -  '  p.  --  p. 


Yn-*'  ^>td  r.  <  yI^It  -  A„  Also,  Z  (y 


j  —  p.+l  P  +1' 

J  Fj  Fj 


(j+1)  . 


A^)  >  0  for 
n  - 


l_.Jj^K:  by  the  similar  property  for  (y^’^'^)  for  *l£Pj>  by  the  definition  of 

K  K 

if  J  p  Finally,  Z  y^-'^^^+P=  i:(Yf-'^)  +  P  -  (K-pOc-  >  Ky.  -  (K-p.  )c. 

J  l“  1-*  JJ 

•  -  Kc;  =  KyP^^^.  The  sequence  is  thus  admissible. 

”  K  J  K  n 

Since  >  p ^ ,  the  above  procedure  must  terminate  in  at  most  K  -  p^^ 

steps.  Moreover,  <  Y^^^  for  l<n<K,  so  that  f^-l  >  fi^lCYi’^^)] 

n  —  n  —  —  Kn  —  Kn 

K  K  K  K 

Assuming  now  that  Z  y  =  Z  A„,  it  is  sufficient  to  show  that  FI  y  >  n  A 

1  n  1  n  1  "  ~  1 

with  equality  if  and  only  if  Yj^  =  A^^,  1  <  n  <  K. 


Define  (3^1^  0  =  ’  Siven  )  such  that  3  ^  =  Y.  and 

n=l  n=l 


^  define  as  follows.  Let  bj  be  the  largest  integer  i 


such  that  Sp^  ^  \-i  necessarily  Aj^  >  Bp^-  Let  a.  be  the  largest  integer 
11  j  j  ^ 


i  such  that  A.  <  Let  A.  =  min  (A,  -  -  A  ),  Define  the 

11  1  b.  b.  ' 

■’  J  J 


a .  a . 

J  J 


sequence  (3p^^P  by 

3(J-1)  =  3U) 


n 


n  i  {a^  ,bj  } 


.  stj’  *  A 

J  J  ^ 


^U^I)  =  gO)  . 

p  ‘p  J 


Cl  carl  V  A  =  C  ^  =  Z  A„. 


n=l 


n 


n=l 


n 


n=l 


n' 


Since  ^  3p^  '*'5 


J 


1  >  ^^d  (y  )  is  non-decreasing,  Bp^^^  3^  Bp'^^^P  Thus, 

a ,  a ,  a^  n 


,(j+n  „  pU) 
'  J 


-  P 


.(j+i)  = 

'■  ■  n  J  “  u  '"a  '■'^b. 

n=l  nfa.,b.  j 

J’  J 


K 


2p'  ^  [ Bp  ^  +  e]  [3p  ^  -  c]  =  n  3p  ^  +  n  bP^  P  -c (3p  ^  -  3p  ^ 


a. 

J 


n=l 


n 


n;^a.  ,b . 
j’  J 


b .  '^a . 

J  J 


<  '  PjP  > 


pU) 


,(j)  = 


n=l 


n 


Y  “  ’'n’  V-  'n  i  'V'  *  'n  \  '  V 


these  relations  holding  for  all  There  are  at  most  K  elements  (y^)  such 

that  and  tlie  number  of  such  elements  is  reduced  by  at  least  one 

whenever  the  sequence  (Bp^^^  is  formed  from  (Bpp.  The  procedure  must 
terminate  in  at  most  K  steps,  and  will  terminate  when  and  only  when  a  sequence 
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